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Scientific  Progress  and  Accomplishments: 

Foreword: 

The  objective  of  this  DURIP  project  was  to  acquire  an  HPC  cluster  and  Mini  CAVE  to  support  research 
projects  involving  simultaneous  parallel  computing  and  parallel  visualization  at  the  University  of  Kansas 
(KU).  We  acquired  HPC  and  visualization  infrastructure  that  together  form  a  single  system  for 
simultaneous  parallel  computing  and  parallel  visualization  research.  The  HPC  cluster  is  composed  of  17 
compute  nodes  with  a  total  of  340  cores,  20  NVIDIA  GPGPU's,  and  14  Intel  Co-Phi  processors.  The 
visualization  infrastructure  is  a  next  generation  tiled  Mini  CAVE  for  semi-immersive  visualization.  Our 
infrastructure  leverages  and  extends  existing  KU  infrastructure  that  is  available  to  the  researchers 
through  the  KU  Advanced  Computing  Facility.  (Further  details  on  the  simultaneous  parallel  computing 
and  parallel  visualization  infrastructure  can  be  found  in  Appendix  A.) 

The  HPC  and  visualization  system  has  supported  research  in  five  Department  of  Defense  (DoD)  mission- 
critical  thematic  areas  involving  simultaneous  parallel  computing  and  parallel  visualization:  unstructured 
meshing,  scientific  visualization,  computational  fluid  dynamics,  elastodynamics,  and  materials  chemistry. 
The  infrastructure  has  enhanced  the  quality  of  research  supported  by  and  of  interest  to  the  DoD.  It  has 
enabled  research  projects  involving:  serial  and  parallel  adaptive  and  moving  mesh  simulations,  scientific 
visualizations,  large  eddy  simulations  of  turbulent  flows  in  gas  engines,  long-range  acoustic  propagation 
simulations,  numerical  modeling  of  nonlinear  nanophotonic  devices,  and  molecular  dynamics 
simulations  of  solidification. 
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Scientific  Progress  and  Accomplishments 


For  each  of  the  seven  investigators  on  the  team,  we  give  a  statement  of  the  problem  studied,  a 
summary  of  the  most  important  results,  and  a  bibliography. 

Investigator:  Weizhang  Huang 

Statement  of  the  problem  studied:  The  porous  medium  equation  models  nonlinear  diffusion  processes 
that  arise  from  several  branches  of  science  such  as  gas  flow  in  porous  medium,  incompressible  fluid 
dynamics,  nonlinear  heat  transfer,  and  image  processing.  Its  numerical  simulation  includes  several 
challenges  including  dealing  with  diffusion  degeneracy  and  moving  free  boundaries  and  necessary  use  of 
dynamical  mesh  adaptation  for  improving  computational  efficiency  and  accuracy.  In  this  project,  we 
have  studied  the  numerical  solution  of  the  porous  medium  equation  using  an  adaptive  moving  mesh 
finite  element  method.  The  method  is  based  on  the  moving  mesh  partial  differential  equation  approach 
and  employs  its  newly  developed  implementation. 

Summary  of  the  most  important  results:  The  method  shows  a  first-order  convergence  for  uniform  and 
arclength-based  adaptive  meshes  and  a  second-order  convergence  for  Hessian-based  adaptive  meshes. 

It  is  also  shown  that  the  method  can  be  used  for  situations  with  complex  free  boundaries,  emerging  and 
splitting  of  free  boundaries,  and  the  porous  medium  equation  with  variable  exponents  and  absorption. 
The  results  obtained  so  far  are  summarized  in  an  article  submitted  to  Journal  of  Computational  Physics 
[1]. 

Bibliography: 

[1]  C.  Ngo  and  W.  Huang,  A  study  on  moving  mesh  finite  element  solution  of  the  porous  medium 
equation,  Journal  of  Computational  Physics  (submitted,  under  revision) 

Investigator:  Suzanne  Shontz 

Statement  of  the  problem  studied:  Parallel  mesh  quality  improvement  is  needed  whenever  meshes  of 
low  quality  arise  in  computationally-intensive  simulations.  An  example  of  such  problems  is  the  direct 
numerical  simulations  for  fluid-structure  interactions.  In  particular,  mesh  quality  improvement  methods 
improve  the  efficiency  and  accuracy  of  the  associated  numerical  PDE  algorithm  and  also  maintains  its 
stability.  There  are  very  few  parallel  variational  methods  for  mesh  quality  improvement.  Therefore,  we 
study  effective  parallelization  strategies  for  a  variational  mesh  quality  improvement  method  proposed 
by  Huang  and  Kamenski  [2,3], 

Summary  of  the  most  important  results4:  We  have  developed  a  parallel  variational  mesh  quality 
improvement  algorithm  designed  for  distributed  memory  machines  (Figure  1).  The  new  parallel 
algorithm  is  based  on  the  sequential  variational  mesh  quality  improvement  method  proposed  by  Huang 
and  Kamenski  [2,3],  To  distribute  the  workload  among  processors,  our  parallel  implementation  employs 
a  partition  of  the  mesh,  which  is  generated  using  METIS.  The  algorithm  solves  an  ordinary  differential 
equation  on  each  interior  node  of  each  region  created  by  the  partition  of  the  mesh.  For  the  nodes  that 
belong  to  the  shared  boundaries,  only  a  partial  ODE  solution  is  calculated  initially,  as  these  nodes  are 
shared  among  cores.  The  ODE  solution  at  the  shared  nodes  is  completed  by  adding  the  partial  solutions 
of  each  processor.  This  operation  is  performed  using  non-blocking  MPI  collective  communication 
instructions  to  overlap  communication  with  computation.  We  use  tetrahedral  meshes  with  different 
sizes  to  test  the  strong  and  weak  scaling  of  our  implementation.  The  numerical  experiments,  performed 


using  up  to  128  processors,  demonstrate  the  excellent  strong  scalability  of  our  implementation  when 
tested  on  meshes  with  80M  and  40M  elements  (Figure  2).  The  weak  scaling  results  are  typical  of  those 
obtained  on  unstructured  meshes. 

In  the  future,  we  plan  to  visualize  our  parallel  adaptive,  moving  meshes  in  the  Mini-CAVE  environment 
which  will  help  us  in  understanding  the  performance  of  our  algorithms. 


Fig.  1:  Average 
mesh  quality  vs. 
number  of  iterations 
for  an  80M  element 
tetrahedral  mesh. 


Fig-  2:  (a)  Total  runtime  and  (b)  speedup  for  the 
Parallel  VMQI  algorithm  for  the  same  80M  element 
tetrahedral  mesh. 


Bibliography: 

[2]  W.  Huang,  L.  Kamenski,  H.  Si,  Mesh  smoothing:  an  MMPDE  approach.  Research  note  at  the  24th  Int. 
Meshing  Roundtable  (2015). 

[3]  W.  Huang,  L.  Kamenski,  A  geometric  discretization  and  a  simple  implementation  for  variational  mesh 
generation  and  adaptation,  J  Comput  Phys,  Vol.  301,  pp.  322-337,  (2015). 

[4]  M.  Lopez,  S.M.  Shontz,  and  W.  Huang,  An  efficient  parallel  implementation  for  a  variational  mesh 
quality  improvement  method,  To  be  submitted  to  Engineering  with  Computers,  November  2016. 

Investigator:  James  Miller 

Statement  of  the  problem  studied:  The  problem  under  investigation  was  the  determination  of  the  best 
way  to  support  stereo  rendering  on  the  mini-CAVE  display  environment. 

Summary  of  the  most  important  results:  A  basic  OpenGL  graphics  framework  developed  by  Dr.  Miller 
and  used  for  both  teaching  and  research  projects  was  modified  to  support  stereo  rendering  on  the  mini- 
CAVE  display  environment.  A  review  of  the  literature  was  performed,  which  resulted  in  three  variations 
for  stereo  view  matrix  generation.  Two  of  these  variations  have  been  previously  described  in  various 
papers.  Two  other  variations  will  also  be  examined.  In  addition,  several  heuristics  for  establishing  good 
stereo  viewing  parameters  will  be  pursued.  Some  of  these  heuristics  for  obtaining  good  stereo  viewing 
performance  have  been  described  in  the  literature,  whereas  others  are  independent  ideas  to  be 
pursued.  An  evaluation  will  be  performed  in  which  it  will  be  determined  how  the  variations  and 
heuristics  compare  to  one  another  so  that  optimal  ones  can  be  quickly,  and  hopefully  automatically, 
generated. 


Bibliography:  N/A 
Investigator:  ZJ  Wang 

Statement  of  the  problem  studied:  We  employed  the  infrastructure  to  perform  large  eddy  simulations 
for  turbulent  flows  in  a  gas  turbine  engine. 

Summary  of  the  most  important  results:  The  results  are  compared  with  experimental  data,  and  very 
good  agreement  has  been  achieved.  A  movie  is  here:  http://dept.ku.edu/~cfdku/LES  turbine  Q  sch.gif 

Two  papers  have  been  published  summarizing:  future  directions  of  high-fidelity  computational  fluid 
dynamics  for  aero-thermal  turbomachinery  [5]  and  large  eddy  simulations  results  based  on  the  FR/CPR 
method  [6], 

Bibliography: 

[5]  G.M.  Laskowski,  J.  Kopriva,  V.  Michelassi,  S.  Shankaran,  U.  Paliath,  R.  Bhaskaran,  Q.  Wang,  C.  Talnikar, 
Z.J.  Wang,  F.  Jia,  Future  Directions  of  High-Fidelity  CFD  for  Aero-Thermal  Turbomachinery  Research, 
Analysis  and  Design,  AIAA-2016-3322. 

[6]  Z.J.  Wang  and  Y.  Li,  Recent  Progresses  in  Large  Eddy  Simulations  with  the  FR/CPR  Method,  Ninth 
International  Conference  on  Computational  Fluid  Dynamics  (ICCFD9),  Istanbul,  Turkey,  July  11-15,  2016, 
ICCFD9-2016-149. 

Investigator:  Zhaoquan  Charlie  Zheng 

Statement  of  the  problem  studied:  Time-domain  simulations  of  ultrasound  propagation  with  fractional 
Laplacian  were  conducted.  Similarly,  physical  simulations  of  sound  propagation  from  aircraft  to  close 
proximity  to  the  ground  were  performed. 

Summary  of  the  most  important  results:  With  the  help  of  the  computer  facility  purchased  using  the 
grants,  the  following  DoD  related  computational  research  projects  have  been  conducted: 

Time-domain  simulation  of  ultrasound  propagation  with  fractional  Laplacian:  The  simulation  is 
developed  for  the  purpose  of  simulating  ultrasound  propagation  through  biological  tissues.  The 
simulation  is  based  on  the  time-domain  conservation  laws  with  the  governing  equations  for  acoustic 
pressure  and  velocity,  with  frequency  dependent  absorption  and  dispersion  effects.  We  use  forward 
differencing  for  velocity  and  backward  differencing  for  pressure  on  the  non-fractional  derivative  operator 
terms  in  spatial  discretization.  The  fractional  Laplacian  operators  are  treated  as  Riesz  derivatives.  The 
shifted  standard  Grunwald  approximation  method  is  used  to  solve  fractional  derivative  operator  terms. 

To  accommodate  complicated  biological  tissue  geometries,  an  immersed  boundary  method  is  developed 
that  enables  a  Cartesian  computational  grid  mesh  to  be  used.  The  results  are  compared  with  those  for  a 
non-absorption  homogeneous  medium  to  discuss  absorption  and  dispersion  effects  of  biological 
material.  Figure  3(a)-(d)  are  pressure  contours  at  four  different  moments.  Sound  wave  propagation  and 
attenuation  through  the  biological  bone  model  can  be  clearly  observed.  Reflections  can  be  observed 
when  the  ultrasonic  plane  waves  enter  the  bone  medium,  due  to  the  absorption  and  dispersion  of  the 
waves  inside  the  bone.  In  Figure  3(d),  it  shows  acoustic  pressure  of  the  plane  wave  dramatically 
attenuated  by  the  lossy  bone  media  comparing  with  the  original  waves  in  3(a).  Meanwhile  there  are 
several  reflected  waves  remaining  inside  the  bone.  This  research  was  partly  supported  by  ARL  under  a 
cooperative  agreement  W911-NF-14-2-0077. 


Fig.  3  :  Pressure  contours  of  ultrasound  propagation  at  different  moments: 
(a)  t=12ps  (b)  t  =20ps  (c)  t  =  24ps  (d)  t  =  30ps. 


Physical  simulation  of  sound  propagation  from  aircraft  in  close  proximity  to  the  ground:  This  study  is 
to  investigate  sound  propagation  from  aircraft  to  the  environment.  The  sound  propagation  is  simulated 
based  on  the  physical  time  and  space,  which  is  a  time-domain  simulation  in  a  physical  spatial  domain. 
The  investigation  will  be  focused  on  the  physics  that  is  usually  difficult  to  reveal  using  frequency-domain 
simulations.  The  effects  to  be  investigated  include  source  characteristics  and  motion  of  aircraft, 
influence  of  local  environment  such  as  ground  roughness  and  impedance,  diffraction  caused  by  local 
structures  such  as  terrain,  buildings,  bushes  and  trees.  Because  the  special  interest  of  the  study  is  for 
aircraft  in  close  proximity  to  the  ground,  such  as  UAVs,  these  effects  are  more  significant  for  sound 
propagation  than  for  the  cases  when  aircraft  are  at  higher  altitude  from  the  ground.  Examples  of  source 
recognition  using  the  simulation  results  are  discussed.  Figure  4  shows  the  Doppler  effects  caused  by  the 
aircraft  motion  both  in  free-space  propagation  and  with  the  rigid  ground  reflection.  The  motion  of  the 
airplane  causes  the  harmonic  frequency  at  300  Hz  to  shift  both  higher  and  lower.  The  Doppler  shifts  are 
cause  by  the  airplane  flying  towards  and  away  from  the  receiver,  with  a  theoretical  estimate  of 

(l  ±  -)  /,  where  V  is  the  airplane  flying  speed  at  50  m/s,  c  is  the  speed  of  sound  at  340  m/s,  and  f  is  the 

harmonic  source  frequency  at  300  Hz.  The  plus  shift  is  when  the  airplane  is  flying  towards  the  receiver 
location  and  the  minus  is  when  the  airplane  is  flying  away  from  the  receiver  location.  The  resultant 
higher  and  lower  frequencies  based  on  this  formula  are  respectively  344  Hz  and  256  Hz.  These 
frequency  values  are  very  close  to  the  two  peak  frequencies  for  the  moving  source  results  shown  in  Fig. 
4  for  both  free  space,  Fig.  4(a)  and  with  a  rigid  ground,  Fig.  4(b).  This  research  was  partly  supported  by 
ARL  under  a  cooperative  agreement  W911-NF-14-2-0077. 
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Fig.  4  :  Effects  of  moving  source  on  sound  pressure  power-spectrum  density  for  an  airplane  flying  at  50  m/s  with 
the  source  represented  by  a  harmonic  Gaussian  source  at  the  frequency  of  300  Hz,  with  the  receiver  location  the 
same  altitude  as  the  airplane  flight  path:  (a)  free  space  propagation;  (b)  a  rigid  ground. 
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Investigator:  Alessandro  Salandrino 

Statement  of  the  problem  studied:  Metamaterials  are  artificially  engineered  media  which  are 
structured  at  a  deeply  sub-wavelength  length-scale  to  provide  a  macroscopic  electromagnetic  response 
with  properties  not  found  in  natural  materials.  Examples  include  negative  index  metamaterials  [7,  8], 
hyperbolic  dispersion  media  [9]  or  epsilon/mu-near-zero  materials  [10].  The  analysis  and  design  of 
metamaterial-based  devices  is  especially  challenging  from  the  computational  point  of  view  because  of 
the  length-scales  involved:  while  the  devices  themselves  have  dimensions  of  the  order  of  the  relevant 
wavelength,  the  structures  which  provide  the  desired  functionalities  are  much  smaller  and  require  an 
extremely  fine  discretization.  The  availability  of  the  high  performance  computing  facilities  acquired 
under  the  US  Army  Research  Office  grant  W911-NF-1510377  has  proven  essential  in  order  to  design  and 
simulate  novel  nonlinear  nanophotonic  devices. 

In  this  project  so  far  we  concentrated  on  the 
analysis  and  design  of  nanophotonic  modulators 
with  optical  nonlinearities  originating  from 
voltage-gated  free-carrier  injection  in  Indium- 
Tin-Oxide  (ITO)  metal-oxide-semiconductor 
(MOS)  structures  (Fig.  5).  Such  devices  require 
the  concurrent  modeling  and  optimization  of  the 
following  characteristics  (with  the  corresponding 
relevant  length-scales: 

1.  Electronic  properties  of  the  materials 
(sub-nm  length-scale) 

2.  Electrostatic  properties  of  the  MOS 
structure  (sub-pm  length-scale) 


Fig.  5.  ITO  MOS  nanophotonic  modulator  layout. 


3.  Electromagnetic  and  modal  properties  of  the  device  in  its  entirety  (pm  length-scale) 


The  main  modeling  challenges  arise  in  combining  the  electronic  modeling  with  the  other  two  aspects,  due 
to  the  large  difference  in  characteristic  lengths.  Moreover,  any  modification/optimization  of  the 
geometry  requires  a  reevaluation  of  all  of  the  three  design  components  -  electronic,  electrostatic,  and 
electromagnetic. 

The  main  goal  of  this  effort  was  to  fully  exploit  the  unique  properties  of  ITO  to  design  a  photonic  device 
with  enhanced  functionalities.  ITO  is  a  degenerately  doped  semiconductor  with  free-carrier  concentration 
that  can  be  tuned  by  controlling  the  concentration  of  oxygen  vacancies  and  interstitial  metal  dopants.  In 
addition  to  doping,  carrier  concentration  can  also  be  electrically  tuned.  Near-unity  index  changes  by 
carrier  injection  in  ITO  have  been  recently  reported  [11,  12].  Such  large  refractive  index  changes  occur 
within  a  charge-accumulation  layer  induced  by  an  electrostatic  bias  at  the  interface  between  ITO  and  an 
insulating  medium  (in  our  case  silicon  dioxide).  Modeling  the  electromagnetic  behavior  of  the  ITO  in  the 
accumulation  layer  poses  challenges  because  a  strong  concentration  gradient  occurs  within  a  sub¬ 
nanometer  thickness  -  i.e.  over  a  length-scale  which  is  three  orders  of  magnitude  below  the  intended 
wavelength  of  operation  (~1550nm).  If  accurately  modeled  and  understood,  such  free-carrier  effects  in 
ITO  can  be  effectively  exploited  to  design  ultra-compact  photonic  electro-absorption  modulators. 

Summary  of  the  most  important  results:  In  the  span  of  one  year  since  the  beginning  of  this  project 
under  the  US  Army  Research  Office  grant  W911-NF-1510377  support  we  have  successfully  addressed 
most  of  the  research  tasks  stated  in  the  previous  section,  with  the  following  main  results: 

1.  We  designed  and  optimized  in  all  its  aspects  (electronic,  electrostatic,  and  electromagnetic)  an 
ITO-based  nanophotonic  modulator  architecture. 

2.  Based  on  the  results  that  we  obtained  we  introduced  the  concept  of  modal-dichroism  in 
nanophotonics,  which  paves  the  way  towards  a  novel  class  of  ultra-compact  mode-multiplexed 
nanophotonic  devices. 

3.  We  have  reported  our  results  in  the  journal  article  [7]  and  we  acknowledged  US  Army  Research 
Office  support. 

The  results  obtained  so  far  have  opened  a  new  research  line  in  Dr.  Salandrino's  group,  which  will  likely 
lead  to  additional  novel  device  concepts  in  the  months  to  come. 
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Investigator:  Brian  Laird 

Statement  of  the  problem  studied:  Most  solidification  occurs  via  heterogeneous  nucleation,  but 
atomistic  studies  of  this  technologically  important  phenomenon  are  rare.  Using  the  GPU  resources 
provided  by  the  DURIP  grant,  we  have  examined  the  effect  of  interfacial  structure  and  orientation  on 
the  heterogeneous  nucleation  of  Pb  crystals  from  the  supercooled  melt  at  a  Cu  substrate  using 
molecular-dynamics  (MD)  simulation,  as  implemented  in  the  highly  parallel  simulation  code  LAMMPS.  In 
a  previous  work  studying  the  Cu/Pb  solid-liquid  interface  with  MD  simulation  [14],  we  observed  that  the 
structure  of  the  Cu(lll)  and  Cu(100)  interfaces  were  significantly  different  at  625K,  just  above  the  Pb 
melting  temperature  (618K  for  the  model).  The  Cu(100)  interface  exhibited  significant  surface  alloying  in 
the  crystal  plane  in  contact  with  the  melt.  In  contrast,  no  surface  alloying  was  seen  at  the  Cu(lll) 
interface;  however,  a  prefreezing  layer  of  crystalline  Pb,  2-3  atomic  planes  thick  and  slightly  compressed 
relative  to  bulk  Pb  crystal,  was  observed  to  form  at  the  interface.  Because  of  the  large  difference  in 
interfacial  structure  between  the  (100)  and  (111)  interfaces,  this  system  provides  an  excellent  test-bed 
to  study  the  orientation  and  surface  structure  dependence  of  heterogeneous  nucleation. 

Summary  of  the  most  important  results:  15We  observe  that  the  Cu(lll)  interface  the  prefreezing  layer 
is  no  longer  present  at  750K,  but  surface  alloying  in  the  Cu(100)  interface  persists.  In  a  series  of 
undercooling  MD  simulations,  heterogeneous  nucleation  of  fee  Pb  is  observed  at  the  Cu(lll)  interface 
within  the  simulation  time  (5  ns)  at  592K  -  a  26K  undercooling.  Nucleation  and  growth  at  Cu(lll) 
proceeded  layer-wise  with  a  nearly  planar  critical  nucleus.  Quantitative  analysis  yielded  heterogeneous 
nucleation  barriers  that  are  more  than  two  orders  of  magnitude  smaller  than  the  predicted 
homogeneous  nucleation  barriers  from  classical  nucleation  theory.  Heterogeneous  nucleation  was 
considerably  more  difficult  on  the  Cu(100)  surface  alloyed  substrate.  An  undercooling  of  approximately 
170K  was  necessary  to  observe  nucleation  at  this  interface  within  the  simulation  time.  From  qualitative 
observation,  the  critical  nucleus  showed  a  contact  angle  with  the  Cu(100)  surface  of  over  90°,  indicating 
poor  wetting  of  the  Cu(100)  surface  by  the  nucleating  phase,  which  according  to  classical  heterogeneous 
nucleation  theory  provides  an  explanation  of  the  large  undercooling  necessary  to  nucleate  on  the 
Cu(100)  surface,  relative  to  Cu(lll),  whose  surface  is  more  similar  to  the  nucleating  phase  due  to  the 
presence  of  the  prefreezing  layer.  The  GPU  resources  were  instrumental  in  completing  this  work  due  to 
the  computational  intensity  of  collecting  nucleation  statistics. 

Current  and  future  work:  We  will  continue  to  use  the  GPU  resources  provided  by  the  DURIP  grant  to 
study  other  metal-metal  and  metal-insulator  solid-liquid  interfaces.  Two  specific  systems  currently 
under  study  are  gallium-aluminum,  which  is  an  important  test  case  to  study  the  phenomenon  of  liquid- 
metal  embrittlement,  and  alumina-aluminum,  a  system  for  which  experimental  data  is  available  for  the 
structure  of  the  solid-liquid  interface  allowing  for  a  rare  comparison  of  simulation  with  experiment  for 
solid-liquid  interfaces.  There  is  potential  in  these  systems  to  also  use  the  Phi-based  CPUs  provided  in  the 
grant,  but  currently  the  Phi  version  of  LAMMPS  is  very  limited,  but  should  be  available  for  the  systems 
here  in  the  near  future.  For  the  alumia-aluminum  systems,  the  visualization  capabilities  made  possible 
with  the  Mini-CAVE  should  also  prove  useful. 
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Appendix  A:  Equipment  Purchased 


HPC  Equipment: 

Quantity  (10)  Standard  Compute  Unit  (SCU)  with  (2)  NVIDIA  K80  GPGPUs 
Dell  PowerEdge  R730 

Mellanox  ConnectX-3,  Single  Port,  VPI  FDR,  QSFP+  Adapter 
Broadcom  5720  QP  1Gb  Network  Daughter  Card 

(2)  Intel  Xeon  E5-2680  v3  2.5GHz,  30M  Cache,  9.60GT/s  QPI,  Turbo,  HT,  12C/24T  (120W) 
Max  Mem  2133  MHz 

128GB  RDIMM,  2133  MT/s,  Dual  Rank,  x4  Data  Width 
1TB  7.2K  RPM  SATA  6Gbps  3.5in  Hot-plug  Hard  Drive 
600GB  Panasas  network  storage 


Quantity  (7)  Standard  Compute  Unit  (SCU)  with  (2)  Intel  Phi  7120p  Accelerators 
Dell  PowerEdge  R730 

Mellanox  ConnectX-3,  Single  Port,  VPI  FDR,  QSFP+  Adapter 
Broadcom  5720  QP  1Gb  Network  Daughter  Card 

(2)  Intel  Xeon  E5-2680  v3  2.5GHz,  30M  Cache,  9.60GT/s  QPI,  Turbo,  HT,  12C/24T  (120W) 
Max  Mem  2133  MHz 

128GB  RDIMM,  2133  MT/s,  Dual  Rank,  x4  Data  Width 
1TB  7.2K  RPM  SATA  6Gbps  3.5in  Hot-plug  Hard  Drive 
600GB  Panasas  network  storage 


$148,935.70 


$89,104.75 


Total: 


$238,040.45 


Visualization  Equipment: 


ITEM  QTY  COST/UNIT 


Mechdyne  Next  Generation  Ultra-High-Definition  Mini  CAVE  1  $245,919 

System 

•  Next  GenerationTiled  Mini  CAVE:  Six  (6)  passive  stereo  46"  1  Included 

displays,  One  (1)  moveable  cart  with  integrated  equipment 

rack,  travel  cases. 

•  Tracking  System:  ART  SMARTTRACK  with  Wand  and  Head  1  Included 

Tracker  markers.  Includes  one  spare  wand  w/  tracking  target 

and  one  spare  head  tracking  target 

•  Laptop  Wired  Connection  Point ;  Extron  Pro  Scalar  with  Auto  1  Included 

Aspect  Ratio  Control,  VGA,  DVI,  HDMI,  or  DisplayPort 

Connection. 

•  3D  System:  Twelve  (12)  Pairs  Passive  3D  Glasses  from  the  1  Included 

display  manufacturer. 

•  USB  Switching  and  Extension:  One  (1 )  Wireless  Keyboard  and  1  Included 

Mouse;  One  (1)  1RU  Rack -mounted  Pull-Out  19" 

Monitor/Keyboard/Mouse  Drawer. 

•  Main  Rendering  Workstation:  Minimum  Specifications  Include:  1  Included 

*  4  RU  Tower 

‘2  CPU,  Xeon.  8core/16thread.  3.2GHz  (E5-2667v3) 

‘128GB  RAM 
*SSD,  2.5",  512GB 
‘HDD,  3.5”,  4TB 

*  ODD.  BD/DVD/CD  read/write 

*  OS,  Microsoft  Windows  7  Professional  x64  SP1 

*  Rack  mounting  rails 

*  4  GPU.  NVIDIA  Quadra  M5000  (8GB) 

*  1  GSU,  NVIDIA  K-Series  Quadra  Sync  Kit 


•  Cabling  Lot,  Adaptors,  Terminations,  and  Consumables  1  Included 

•  Mechdyne  Design  and  Integration:  AV  Design,  Mechanical  1  Included 

Design,  Applications  Engineering,  Project  Management, 

Staging.  &  Installation 

•  Mechdyne  Travel  and  Living  Total:  1  Included 


EXTENDED 

$245.91 9 

Included 

Included 

Included 

Included 

Included 

Included 


Included 

Included 

Included 


Mechdyne  Shipping  and  Handling: 


1 


Included 


Included 


Mechdyne  Software 


•  Conduit  configured  for  the  Mini-CAVE  system  1  Included 

•  Supports  3D  passive  stereo  viewing  and  user 
perspective  with  the  tracking  system. 

•  Conduit  for  Visit.  TecPIot,  TetGen/TetView,  Google 
Earth,  Sketchup 

•  Includes  one  (1)  year  of  remote  support  and  upgrades 


•  getReal3D  for  Unity  configured  for  the  Mini-CAVE  system  1  Included 

•  Supports  3D  passive  stereo  viewing  and  user 
perspective  with  the  tracking  system. 

•  Includes  one  (1)  year  of  remote  support  and  upgrades 

•  Requires  Unity  Pro  for  development  (not  included) 


•  On-site  software  usage  training  included  (one  day  for  initial  1  Included 

Development  System  and  one  day  for  mini-CAVE  system 


Maintenance  Support 

•  Pricing  for  year  1  1  $7,080 

1  year  Total 


•  1  System  Support  visits-  System  Support  visits  are 
one  day  visits  used  to  provide  quick  unscheduled 
response  to  resolve  system  issues  and  provide  onsite 
parts  replacement.  System  Support  onsite  response  time 
of  72  hours. 

•  Mechdyne  parts  coverage-  Parts  coverage  provides 
warranty  for  all  components  of  your  system. 
Consumables  are  excluded  but  a  quote  for  consumable 
coverage  can  be  provided. 

•  24/7  unlimited  telephone  and  email  support-  Phone  and 
email  support  will  help  resolve  minor  issues  immediately 

•  Access  to  PIVOT-  PIVOT  allows  clients  to  log  into  our 
service  database  and  create  repair  tickets  for  their 
system  issues.  You  then  have  access  to  track  the  status 
of  your  repair  and  see  the  final  report  when  the  repair  is 
complete. 

•  Priority  repair  service  -  Priority  service  is  provided  to  our 
contract  clients.  This  means  you  will  be  serviced  before 
those  who  do  not  have  contracts. 

•  Discounted  labor  rates-  Maintenance  contract  clients 
receive  discounted  labor  rates  for  any  work  provided  that 
is  not  covered  under  the  maintenance  contract. 


•  Other  exclusions,  limitations,  terms,  and  conditions  may 
apply  to  the  Mechdyne  Service  and  Warranty  solutions. 
Please  speak  with  your  account  representative  for  a 
comprehensive  support  proposal. 


Included 


Included 


Included 


$7,080 

$7,080 


Development  System 


•  3D  Display  with  Mounting  Cart:  One  (1 )  60"  UHD  Resolution  1  Included 

(3840x2160)  passive  stereo  display,  One  (1 )  display  mount  cart 

with  castors.  Includes  10  pairs  of  stereo  glasses  from  the 
display  manufacturer. 

•  Spare  Panels:  One  (1)60"  UHD  Resolution  (3840x2160)  1  Included 

passive  stereo  display, 

Note:  The  display  proposed  here  is  a  consumer-grade 
television  display.  It  will  likely  only  have  a  90  day 
warranty  in  a  commercial  application.  This  is  not  a 
professional  grade  panel,  is  not  rated  for  continuous 
use,  and  we  cannot  guarantee  availability  of 
replacements,  parts,  or  related  beyond  one  year  due  to 
rapid  model  changes  of  consumer  grade  flat  panel 
displays.  As  a  result,  we  are  providing  this  one  spare 
panel. 


Total 


Included 


Included 


$252,999 


Appendix  B:  Special  Circumstances  in  Acquisition  of  Infrastructure 


The  Mechdyne  original  proposed  configuration  of  the  Mini-cave  encountered  a  delay  because  the 
company  that  manufactures  the  46"  display  panels,  Planar,  shut  down  their  manufacturing  plant  shortly 
after  the  grant  period  to  upgrade  their  production  line  to  produce  higher  resolution  panels  (from  720p 
to  1080p).  The  original  720p  panels  that  were  quoted  for  the  Mini-cave  system  were  unavailable  due 
to  large  orders  from  customers  already  in  process.  We  were  one  of  the  first  customers  in  line  to 
purchase  the  new  1080p  panels  when  manufacturing  resumed.  The  panels  are  specialized,  non¬ 
consumer  grade  displays  that  cannot  be  replaced  by  standard  panels. 

To  compensate  KU  for  the  delay,  a  temporary,  replacement  Mini-cave  configuration  was  provided  from 
Mechdyne  in  March,  2016,  for  no  additional  charge,  with  a  matrix  of  1  x  3  consumer  grade  60"  4K  3-D 
displays  (the  original  configuration  was  a  2  x  3  matrix  of  46"  3-D  displays).  This  provided  a  comparable 
platform  for  researchers  to  use  for  the  visualization  simulations.  When  manufacturing  resumed  and  the 
Planar  panels  were  available  in  June,  2016,  Mechdyne  built  and  tested  the  Mini-cave  and  delivered  it  to 
KU  the  second  week  of  July,  2016. 

A  second  delay  for  some  of  the  researchers  who  use  a  Linux  environment  for  their  visualizations  was 
due  to  a  mixup  between  the  Mechdyne  sales  team  and  the  quoted  configuration  for  the  Mini-cave.  KU 
was  led  to  believe  there  would  be  support  for  a  dual-boot  operating  system  (Windows  and  Linux) 
and  the  Mechdyne  visualization  software  environment.  After  some  discussions  about  the  mixup,  it  was 
decided  KU  would  install  and  support  the  Linux  operating  system  and  Mechdyne  would  install  and  test 
their  visualization  software  environment,  but  with  only  software  support  for  the  Windows  versions  of 
the  software.  The  configuration  of  the  Linux  environment  by  Mechdyne  encountered  additional  delays 
due  to  compatibility  with  NIVIDIA  drivers  for  the  multi-display  and  multi-GPU  configuration. 


